Papers with automatic prompt engineering
Prompt Engineering a Prompt Engineer (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies indicate that large language models can be meta-prompted to perform automatic prompt engineering, but their potential is limited due to insufficient guidance for complex reasoning in the meta-prompt. |
| Approach: | They propose to infuse three key components into a meta-prompt to guide reasoning . they find prompts that outperform “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K . |
| Outcome: | The proposed method outperforms “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K and outperfies baselines on counterfactual tasks by 6.9%. |
Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering (2026.findings-acl)
Copied to clipboard
Anas Mohamed, Azal Ahmad Khan, Xinran Wang, Ahmad Faraz Khan, Shuwen Ge, Saman Bahzad Khan, Ayaan Ahmad, Ali Anwar
| Challenge: | Direct Preference Optimization (DPO) is an off-policy alternative to RL for automatic prompt engineering, but its token-level regularization leaves semantic inconsistency unchecked as prompts that win higher preference scores can still drift away from the user’s intended meaning. |
| Approach: | They propose a variant of Direct Preference Optimization that preserves semantic consistency while maintaining its simplicity and efficiency. |
| Outcome: | The proposed model outperforms state-of-the-art prompt optimization baselines and several DPO variants on three standard text-to-image prompt-optimization benchmarks and three language models. |